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ABSTRACT 

This study used eye-tracking technology to track students’ eye movements while searching information on the web. The 
research question guiding this study was “Do students with different search performance levels have different visual 
attention distributions while searching information online? If yes, what are the patterns for high and low performing 
searchers?” 14 university students in Macau were invited to search an answer online for the question: “After you clean a 
glass with tap water, why are there always some water drops remaining in the glass surface?” A Tobii Pro X2-30 remote 
eye tracker was used for data collection. Two students with the highest and lowest search task scores were selected for 
comparisons in their online search behavior and eye-movement patterns. The high-performing student worked more in 
revising search queries, and reading and assessing the information in the selected webpages for its relevance. Search 
patterns showed that the high-performing student tended to switch more frequently between search query box and search 
result page whereas the low-performing student tended to switch between the search result and selected web page. This 
implied that the high-performing student was more metacognitively engaged by revisiting and revising search queries to 
improve the quality of search results. 
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1. INTRODUCTION 

Eye-tracking measures can reveal individuals’ cognitive processes, especially attention distributions which 
reflected the use of cognitive and metacognitive strategies to some degree (Lai et al., 2013), for example, in 
equation solving (Susac et al., 2014) and translation tasks (Angelone, 2010). To date, insufficient effort has 
been made to investigate how individuals allocate their attention and use (meta)cognitive strategies to search 
information online, although the research on web search has grown vastly in the past decade. For this 
purpose, this study used eye-tracking technology to track and observe students’ eye movements while 
searching information on the web. The research question guiding this study was “Do students with different 
search performance levels have different visual attention distributions while searching information online? If 
yes, what are the patterns for high and low performing searchers?” 


2. REVELANTWORK 

Eye-tracking methodologies seem particularly promising in online search because gaze can be used as a 
proxy for a user’s attention (Cutrell & Guan, 2007). Ocular indices by eye-tracking techniques enable us to 
determine what information a user is indeed viewing and reading, for how long, and in what order. The 
moment-by-moment observations by eye-tracking provide more details about how users interact with 
information online than other measures such as mouse clicks, query streams or web logs (Cutrell & Guan, 
2007). Efforts have been made in the past years in examining how individuals search, process and assess 
information online by eye-tracking. For example, Joachims et al. (2005) used eye-tracking techniques to 
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characterize how users peruse search results; Pan et al. (2004) investigated the factors that affected web page 
viewing behavior during search; Gerjets, Kammerer, and Werne (2011) examined how individuals evaluated 
search results. 

The above studies provided important insights in understanding how individuals search for information 
on the Web, yet they tended to investigate this process in specific stages, e.g., browsing results, assessing and 
selecting results and etc. The research presented here seeks to obtain a more comprehensive understanding of 
whether and how the searcher regulates the search process as a whole and what cognitive and metacognitive 
strategies were applied to reach their goal. 


3. METHOD 

A group of 14 students from different departments in a public university in Macau (50% females, mean age 
was 23 years old) were invited to a computer station to search an answer online for the question: “After you 
clean a glass with tap water, why are there always some water drops remaining in the glass surface?” All 
participants used IE as the web browser. The students were allowed to finish the task in their own paces. 
Each answer was scored by two research assistants based on the completeness and accuracy of the answer. A 
Tobii Pro X2-30 remote eye tracker was used for data collection. 

Several indices were used as indicators of ocular behaviors. Eye fixations are defined as a spatially stable 
gaze lasting for approximately 200-300 milliseconds, which represents the instances in which information 
acquisition and processing is able to occur (Rayner, 1998). Scanpaths were used to reveal eye-movement 
patterns which connected saccades and fixations. They depicted a complete sequence of fixations and 
saccades within different areas of interest (AOIs). We defined three AOIs for each screen within each task 
and participant: the search box area, search result page and selected webpage. For each AOI, we computed 
eye fixation duration and frequency count. 


4. RESULTS AND DSICUSSION 

Two students with the highest (full score) and lowest scores (zero score) were selected respectively for 
conducting comparisons in terms of their online search behavior and eye-movement patterns. As shown in 
Table 1, the high-performing student gazed for a significantly longer time, with a much higher frequency in 
the search box and selected webpages than the low-performing student. The heatmaps in these two AOIs 
revealed that the high-performing student did not spend much more time in scanning the result page, but 
working more in revising search queries and reading and assessing the information in the selected webpages 
for its relevance to the search problem. This revealed the more effective use of cognitive strategies in 
high-performing searchers: significantly more time allocated to search query construction and information 
evaluation. These two stages in web search seem to present higher chances for search success. 

Table 1. Total eye fixation duration and total eye fixation count for low and high-performing students 

Total Eye Fixation Duration (s) Total Eye Fixation Count 

search box result page selected webpage search box result page selected webpage 
Low 80.9 13.35 131.3 342 34 557 

High 165.3 16.15 483.8 723 55 2195 


Subsequent search behavioral patterns in these two participants showed clear differences in the process of 
online search. The high-performing student tended to switch more frequently between search query box and 
search result page whereas the low-performing student tended to switch between the search result and 
selected web page (see Figure 1 ). This suggests that the high-performing student was more metacognitively 
engaged by revisiting and revising search queries to improve the quality of search results and thus search 
outcomes than the low-performing student. 
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Figure 1. Search behavioural patterns for low and high-performing students. 1 = Construct and revise search queries; 
2 = Scan and evaluate search results; 3 = Select a result and access the webpage 


5. CONCLUSION 

We presented a study using eye tracking techniques to investigate online search strategies. We believe this 
study is only a prelude to examine this issue from a metacognitive perspective. It would be interesting to 
replicate this study with a larger and more diverse sample to verify the patterns we located here. In such way, 
we can further provide suggestions or design interventions to help searchers to improve their online 
information search performance. 
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